Is that your final answer? The effects of neutral queries on children's choices

Preschoolers often switch a response on repeated questioning, even though no new evidence has been provided (Krahenbuhl, Blades, & Eiser, 2009). Though apparently irrational, this behavior may be understood as children making an inductive inferrence based on their beliefs about whether intial responses were correct and the knowledgeability of the questioner. We present a probabilistic model of how the questioners’ knowledge and biases to be positive should affect inferences. The model generates the qualitative prediction that an ideal learner should switch responses more often following a “neutral query” from a knowledgeable questioner than following queries from an ignorant questioner. We test predictions of the model in an experiment. The results show that four-yearold children are sensitive to questioners’ knowledge when responding to a neutral query, demonstrating more switching behavior when the query is provided by a knowledgeable questioner. We conclude by discussing the practical and theoretical implications for cognitive development. When should a learner abandon their current hypothesis in favor of a new one? It is becoming clear that even preschoolaged children rationally update beliefs and generate new explanations following informative evidence (Gopnik, Glymour, Sobel, Schulz, & Danks, 2004; Schulz, Bonawitz, & Griffiths, 2007; Denison, Bonawitz, Gopnik, & Griffiths, 2010; Bonawitz, Schijndel, Friel, & Schulz, 2012; Bonawitz, Fisher, & Schulz, in press). These tasks often involve assessing children’s starting belief state, either presenting the child with new evidence or allowing the child to generate their own evidence, and then eliciting an explanation or judgment. What constitutes “evidence” in these tasks is fairly intuitive; for example, children may be presented with a storybook containing information about two variables that tend to co-occur (Schulz et al., 2007) or they may observe a toy that reacts when certain objects are placed on top of it (Bonawitz, Denison, Gopnik, & Griffiths, 2011). In addition to revising beliefs following correlational evidence and interventions, children also learn from others. Interestingly, even preschool-aged children do not do so indiscriminately; they use information about informants’ knowledge and intent to guide who they trust. For example, children do not trust informants who label familiar objects incorrectly (Koenig & Harris, 2005), who dissent from groups (Corriveau, Fusaro, & Harris, 2009), and even with familiar informants, children update social inferences (Corriveau & Harris, 2009). Similarly, recent research suggests that children use information about informants’ intent to guide inferences (Mascaro & Sperber, 2009; Shafto, Eaves, Navarro, & Perfors, in press). Other research has suggested that even four-year-old children make different causal inferences depending on the social context when evidence is presented: direct instruction by a knowledgeable and helpful informant provides strong contraints on likely hypotheses as compared to accidental information by a not-knowledgeable informant. Even when contrasted with intentional (but not instructional) actions and interrupted demonstrations, preschoolers make stronger inferences about causal structure from direct instruction by leveraging the informant’s knowledge and intent (Bonawitz, Shafto, et al., 2011; Buchsbaum, Gopnik, Griffiths, & Shafto, 2011; Butler & Markman, 2010). These pedagogical assumptions have been captured by probabilistic models (e.g. Shafto & Goodman, 2008; Bonawitz, Shafto, et al., 2011), which suggest a rational account of how learners update their beliefs following evidence generated in the context of teaching. These literatures suggest that preschool children are sophisticated and powerful social learners; they revise their beliefs when evidence is sufficient and use social or instructional contexts to help interpret the strength of the evidence. However, other research suggests that children may abandon hypotheses too capriciously. For example, the effects of repeated questioning on eyewitness testimony in young children have been studied extensively in the context of the judicial system, and work done by Poole and White (1991) found that, in contrast to adults, four-year-olds were more likely to change their responses to repeated yes or no questions. More recently Krahenbuhl et al. (2009) found that children as young as four changed over a quarter of their responses to repeated questions, resulting in a decline in accuracy. In Howie, Sheehan, Mojarrad, and Wrzesinska (2004) four-year-old children were more likely than older children to change responses toward an incorrect answer on repeated questioning. That is, although no additional evidence was provided, by simply asking children the same question a second time, children were very likely to switch their predictions. How might we reconcile these findings with those suggesting that children should only rationally update beliefs following informative evidence? One explanation for this apparently irrational switching behavior is that seemingly neutral questions such as “Is that your final answer?” may provide strong information in certain social contexts. If preschoolers are sensitive to the potential communicative intent behind such queries, the question itself may be a source of evidence about whether an initial guess was correct. Consider a game in which a sticker is hidden under one of two cups. Suppose an informant asks the child which cup they believe the sticker is under and after the child’s initial guess, the teacher asks, “Is that your final answer? Would you like to change your guess?” In which contexts does such a question provide information about the true location of the sticker? Intuitively, it seems obvious that if the questioner does not know the actual location of the sticker, then the repetition of the question provides little additional evidence; however, a knowledgeable questioner might have a good reason for giving the learner with a second chance at answering the question. In these cases, this apparently neutral query is not neutral at all; it is a strong cue to the learner that they should change their answer. In what follows we will explore the idea that even a “neutral” query carries information about the state of the world when a questioner is knowledgeable. We present a probabilistic model that demonstrates how an ideal learner might evaluate such “neutral” queries in scenarios in which the questioner is knowledgeable and scenarios in which she is ignorant. With the model, we evaluate the conditions under which switching guesses is the rational choice for the learner. We then test the basic prediction with preschoolers in an experiment in which the informant’s knowledge or ignorance is made explicit. We suggest that repeated questioning does indeed lead a learner to switch responses and that even preschoolers are sensitive to the knowledge state of others when making such inferences. Modeling learners’ responses to neutral queries Here we consider a model of “neutral queries”. Bayesian inference provides a natural framework in which to consider how an ideal learner should update her beliefs following this kind of information. In Bayesian inference, the learner’s goal is to update their beliefs about hypotheses, h, given data, d, where the degree of belief in a hypothesis given data is denoted P(h|d). These updated posterior beliefs are determined by two factors: the learner’s prior beliefs in hypotheses, P(h), and the probability of sampling the observed data, assuming the hypothesis is true, P(d|h). Specifically, the updated posterior belief in a particular hypothesis is proportional to the product of the prior belief in that hypothesis and the probability of sampling the data given that hypothesis, P(h|d) ∝ P(d|h)P(h). Because we are considering only two hypotheses, we can use Bayes Odds to simplify the problem: P(h1|d) P(h2|d) = P(d|h1)P(h1) P(d|h2)P(h2) , (1) where P(h1|d) is the probability that the sticker is in the first location, given the statement from the informant (“correct”, “incorrect”, “is that your final answer”) and P(h2|d) is the probability that the sticker is in the second location given the statement. It is reasonable to assume that the learner has no a priori assumptions about either location, which allows us to cancel out the prior beliefs (i.e. P(h1) = P(h2), thus P(h1) P(h2) = 1). The main issue is the probability of the statement given the location of the sticker, P(d|h). We can model this likelihood with a simple causal graphical model (see Figure 1). Causal graphical models consist of a structure indicating the causal relationships among a set of variables, where nodes are Figure 1: Graphical model depicting dependencies in cases when the informant is knowledgeable and ignorant. variables and dependence relationships are indicated by arrows from causes to effects. To complete a graphical model, conditional probability distributions give the probability that each variable takes on a particular value given the value of its causes (Pearl, 2000; Spirtes, Glymour, & Schienes, 1993, see Table 1). In our model of the problem, the variables include the actual state of the world (“World”, i.e. location of the sticker), the intention of the speaker (to provide helpful feedback, to avoid negative feedback, etc.), and the possible statements the informer can make (“Correct”, “Incorrect”, “Is that your final answer?”). In the first model (Knowledgeable) the informant is aware of the actual state of the world. As a result, both the true location of the sticker and the intention of the speaker influence the statement given to the learner. In the second model (Ignorant), the informant is not aware of the actual state of the world. As a result, the true state of the world does not influence the statement given to the learner. That is, the state of the world and the information provided are causally independent of each other. The dependence assumptions captured by the graphical model generate predictions about the behavior of learners. In the case of a knowledgeable informant, given information about the actual state of the world and the informant’s statement, a learner can infer something about the informant’s goals (e.g. to provide positive or negative feedback). Given information about the state of the world and the informant’s goals, a learner could also predict (with some probability) the likelihood that the informant would produce different statements. In our problem, given the informant’s goals and the statement provided, the learner can make an inference about Table 1: Conditional probability table for knowledgeable graphical model. Guess “Correct” “Incorrect” “Final answer?” Correct αc βc ≈ 0 1− (αc +βc)≈ 1−αc Incorrect αi ≈ 0 βi 1− (αi +βi)≈ 1−βi the state of the world. A learner could also make more abstract inferences: given information about the goals of the informant, the statement provided, and the actual location of the sticker, a learner could infer which model (Knowledgeable vs Ignorant) best captures the knowledge state of the informant. The specification of the conditional probability distribution provides additional qualitative predictions. In the knowledgeable graphical model, we might reasonably argue that if the child chooses the correct location initially, the speaker is very unlikely to say “incorrect”; in this case P(“incorrect”|correct choice) = βc ≈ 0. Similarly, if the child chooses the incorrect location intially the speaker is very unlikely to say “correct”, P(“correct”|incorrect choice) = αi ≈ 0. Given these intuitive assumptions, we can compare three possible ways biases about the goals of the teacher might play out in the model’s predictions. The first possibility is that the informant is unbiased. Let us consider the case when the informant is knowledgeable. In this case, the unbiased informant is just as likely to say “correct” when the initial guess is correct as “incorrect” when the initial guess is incorrect, αc ≈ βi. If this is the case, then the learner can not infer whether their initial guess is correct or not if they hear the statement “is that your final answer”. This is because P(“final answer?”|correct) = P(“final answer?”|incorrect). That is, the statement “is that your final answer” provides no additional information about the location of the sticker (Equation 1 is approximately equal to 1). Now consider the case where the informant is ignorant. In this case, because the informant has no information about the actual state of the world, the true location is conditionally independent of the statements made by the informant, and the learner cannot make any inferences about the state of the world. Thus, assuming unbiased informants, learners should make the same inferences if asked “is that your final answer” in a knowledgeable condition as if asked “is that your final answer” in an ignorant condition. This model does not predict the degree to which the leaners should switch responses, but it does predict no difference between conditions. A second possibility is that the informant is positively biased. In this model, the knowledgeable informant may be inclined to want to say “correct” following correct initial guesses, but would be reluctant to say “incorrect” following an incorrect initial guess, αc > βi. If this is the case, then the statement “is that your final answer” provides support for the hypothesis that the learner’s initial guess was incorrect because she is more likely to hear “is that your final answer” given an incorrect guess than “is that your final answer” given a correct guess (Equation 1 > 1). Thus, the positively biased model predicts that a learner should show increased switching in a Knowledgeable condition as compared to an Ignorant condition (in which the state of the world is still conditionally independent of the statements and thus does not provide additional information). The third possibility is that the informant is negatively biased. In this model, the informant may be inclined to say “incorrect” following an incorrect initial guess, but would be comparatively reluctant to say “correct” following a correct intial guess, αc < βi. If this is the case, then the statement “is that your final answer” provides support for the hypothesis that the learner’s initial guess was correct (Equation < 1) and the learner should show a decrease in switching responses in the knowledgeable condition as compared to an Ignorant condition. Note that the precise values of α and β are not important for the predictions of this model, but the relationship between these variables drives the predictive differences. We investigate three implications of this model: first, do we replicate the finding that preschoolers tend to switch responses following what might be considered a “neutral query”; second, do preschoolers take the knowledge state of the informant into account when inferring whether or not to switch hypotheses; third, do preschoolers assume that the informant is neutral, positively, or negatively biased when they provide a query? Experiment: Preschoolers’ switching behavior in response to a neutral query To investigate the predictions of our model we invited preschoolers to participate in a game where the goal was to guess the location of a sticker under one of two cups. After their initial guesses, children were given some feedback and the opportunity to change their guess. After two training trials in which the experimenter told the child that their first guess was either correct or incorrect, children were given three test trials. In the test trials the experimenter asked the child “Is that your final guess?” after children’s initial guesses. Some children participated in a condition in which the experimenter looked under the cups before generating the query and others participated in a condition in which the experimenter did not look before the query. The critical measure is simply on what percentage of trials children switched their prediction to the other cup by condition.